kourouklidesfandomcom-20200213-history
Optimization
This page contains resources about Mathematical Optimization and Operations Research. More specific information is included in each subfield. Subfields and Concepts See Category:Optimization for some of its subfields. * Convex Optimization ** Linear Programming *** Big M method *** Simplex Algorithm * Quadratic Programming * Nonlinear Programming ** Karush–Kuhn–Tucker (KKT) conditions * Combinatorial Optimization ** Integer Programming ** Dynamic Programming ** Greedy Algorithm * Variational Analysis‎ ** Calculus of variations * Optimal Control Theory * Lagrange Multipliers * Iterative Methods ** Levenberg–Marquardt Algorithm ** Iteratively Reweighted Least Squares ** Nonlinear Least Squares ** Gradient Descent / Steepest Descent ** Broyden–Fletcher–Goldfarb–Shanno (BFGS) Algorithm * Stochastic Optimization ** Stochastic Gradient Descent (SGD) ** AdaDelta ** AdaGrad ** Adam ** Gradient Descent with Momentum Backpropagation ** Nesterov’s Accelerated Gradient ** Resilient Backpropagation (Rprop) ** RMSprop ** Follow the regularised leader (FTRL) ** Bayesian Optimization ** Stochastic Gradient Langevin Dynamics (SGLD) ** Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) ** Stochastic Gradient Riemann Hamiltonian Monte Carlo (SGRHMC) ** Stochastic Gradient Markov Chain Monte Carlo (SGMCMC) ** Stochastic Gradient Nose-Hoover Thermostat (SGNHT) ** Relativistic Stochastic Gradient Descent / Relativistic Monte Carlo ** Particle Mirror Descent (PMD) * Metaheuristics ** Evolutionary Algorithms ** Simulated Annealing * Inverse Problems ** Regularization *** Tikhonov regularization / Ridge regression *** Least absolute shrinkage and selection operator (LASSO) *** Early stopping *** Total variation regularization Online Courses Video Lectures * Optimization Course by Michael Zibulevsky * Convex Optimization I by Stephen P. Boyd * Convex Optimization II by Stephen P. Boyd * Discrete Optimization by Professor Pascal Van Hentenryck - Coursera * Linear and Discrete Optimization by Friedrich Eisenbrand - Coursera * Advanced Optimization and Randomized Methods by A. Smola and S. Sra * Optimization, Learning and Systems by Martin Jaggi Lecture Notes Introductory * Practical Optimization: A Gentle Introduction by John W. Chinneck - a very introductory course * Lecture Notes on Optimization by Pravin Varaiya * Lecture Notes on Engineering Optimization by Fraser J. Forbes and Ilyasse Aksikas * Optimization by Geoff Gordon and Ryan Tibshirani * Optimization: An introduction by A. Astolﬁ * Optimization for Differential Equations by Martin Berggren and Krister Svanberg * Quantitative Methods by Michael Trick * Operations Research. Linear Programming by Ana Zelaia Jauregi Specialized * Combinatorial Optimization by Santosh Vempala * Discrete Optimization for Vision and Learning by Nikos Komodakis and Pawan Kumar * Lectures on Modern Convex Optimization by Aharon Ben-Tal and Arkadi Nemirovski * Numerical Optimization by Christopher Griffin * Lecture Notes on Continuous Optimization by Kok Lay Teo and Song Wang * Introduction to Online Optimization by Sébastien Bubeck * Lecture Notes on Online Learning by Alexander Rakhlin * EECS260 Optimization by Miguel Á. Carreira-Perpiñán * Optimization Techniques in Engineering by Alan R. Parkinson and John D. Hedengren * Multidisciplinary System Design Optimization by Olivier de Weck and Karen Willcox * ORF523: Advanced Optimization by Sébastien Bubeck Books Introductory * Chong, E. K., & Zak, S. H. (2013). An Introduction to Optimization (Vol. 76). John Wiley & Sons. * Luenberger, D. G., & Ye, Y. (2008). Linear and Nonlinear Programming (Vol. 116). Springer. Specialized * Fletcher, R. (2013). Practical Methods of Optimization. John Wiley & Sons. * Luke, S. (2009). Essentials of Metaheuristics (Vol. 113). Raleigh: Lulu. * Press, W. H. (2007). Numerical Recipes 3rd edition: The Art of Scientific Computing. Cambridge University Press. * Nocedal, J., & Wright, S. J. (2006). Numerical Optimization. Springer. * Boyd, S. P., & Vandenberghe, L. (2004). Convex Optimization. Cambridge university Press. *Nesterov, Y., & Nesterov, I. E. (2004). Introductory Lectures on Convex Optimization: A Basic Course (Vol. 87). Springer. *Saad, Y. (2003). Iterative Methods for Sparse Linear Systems. Siam. *Vogel, C. R. (2002). Computational Methods for Inverse Problems (Vol. 23). Siam. *Kelley, C. T. (1999). Iterative Methods for Optimization (Vol. 18). Siam. *Dennis Jr, J. E., & Schnabel, R. B. (1996). Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Vol. 16). Siam. Software See List of Optimization Software for the complete list. See also * Sparse Coding * Kernel Methods * Estimation Theory * Machine Learning * Computer Vision Other Resources *NEOS Optimization Guide *COIN-OR - Computational Infrastructure for Operations Research *Decision Tree for Optimization Software Links to optimization source codes *Mathematical Programming Glossary *Optimization Related Links *OPTML - NIPS Workshop on Optimization for Machine Learning Category:Optimization